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ABSTRACT 

The  United  States  Army  is  in  the  midst  of  a  major 
transformation.  The  Future  Force  must  be  highly  mobile, 
agile,  and  lethal  to  ensure  its  dominance  in  the  future 
battlefield.  This  dominance  is  reliant  on  the  ability  to  see 
and  understand  first  (situational  awareness).  Persistent 
and  pervasive  sensing  and  processing,  coupled  with 
greatly  increased  speed  of  information  flow,  information 
assimilation,  and  decisive  action,  at  and  between  all 
levels  of  our  force,  are  necessary  to  fulfill  this 
requirement. 

The  United  States  Army  is  most  vulnerable  in  urban 
terrain.  This  highly  constrained,  complex  environment 
presents  a  significant  challenge  to  US  forces,  particularly 
dismounted  infantry  and  military  intelligence  because 
opposing  force  activity  is  readily  masked  or  obscured  by 
background  noise  (commerce,  schools,  religious  activity 
etc.).  This  paper  proposes  an  intelligent  interaction 
between  the  digitized  dismounted  units  and  military 
intelligence,  for  the  significant  benefit  to  both. 

1.  INTRODUCTION 

The  United  States  Army  is  most  vulnerable  in  urban 
terrain.  This  highly  constrained,  complex  environment 
presents  a  significant  challenge  to  US  forces,  particularly 
dismounted  infantry.  In  order  to  enhance  situational 
awareness  in  the  urban  terrain,  future  dismounted  squads 
will  be  augmented  by  networked  mobile  and  static 
sensing  assets;  unattended  ground  sensors,  small 
unmanned  ground  vehicles,  a  robotic  Multifunction 
Utility/Logistics  Equipment  Vehicle  (MULE),  and  small 
unmanned  ah  vehicles.  A  future  dismounted  force 
engaged  in  urban  warfare  is  depicted  in  Figure  1 . 

Each  warfighter  will  act  as  a  mobile  sensor  and 
communications  node  since  they  will  be  equipped  with 
sensors  such  as  GPS,  imagers,  laser  rangefinders,  and 
will  be  networked  locally.  This  organic  sensor  and 
communications  augmentation  of  dismounted  squads  is 
consistent  with  the  Future  Combat  System  (FCS)  and 
Future  Force  Warrior  concepts  [1].  This  concept  has  also 
been  demonstrated  by  Army  Research  Laboratory  in  the 
Warrior’s  Edge  program.  This  complex  urban 
environment  also  poses  challenges  to  Military 
intelligence  [2]  since  opposing  force  activity  is  readily 
masked  by  buildings  or  obscured  by  background  noise 


(commerce,  schools,  religious  activity  etc.).  This  sensor 
and  communications  network  augmentation  will  provide 
a  greatly  increased  level  of  information  to  the  warfighter. 


Figure  1.  Future  Urban  Warfare  Scene 


In  order  to  effectively  utilize  (lower  uncertainty  and 
enhance  situational  understanding)  these  streams  of 
organic  sensory  data,  it  is  necessary  to  fuse  them  before 
presentation  to  the  warfighter.  These  fusion  applications 
remove  redundancies,  correlate,  aggregate,  and  integrate 
sensor  information  with  context  such  as  high  resolution 
maps. 

The  communications  and  processing  enhancements 
embedded  in  a  small  organic  fusion  station  will  also 
support  effective  linkages  to  other  information  nodes 
(local  and  global).  This  augmentation  will  enable  the 
extension  of  the  network  centric  warfare  [3]  to  the 
dismounted  infantry,  effectively  linking  the  future 
dismounted  warfighter  (along  with  organic  sensors)  to 
tactical  networks  and  particularly  to  Global  Military 
Intelligence  (assuming  security  policy  will  allow).  This 
network  coupling  of  the  dismounted  warfighter  (local 
world)  and  global  Military  Intelligence  (global  world) 
can  significantly  improve  the  situational  understanding 
of  both  the  dismounted  infantry  and  Military 
Intelligence. 

As  an  example,  consider  a  dismounted  force  preparing  to 
secure  a  portion  of  a  town.  Twenty  (20)  minutes  prior  to 
entering  the  town  they  receive  a  report  that  there  are  ten 
(10)  communication  channels  that  have  become  active  in 
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the  last  fifteen  (15)  minutes  in  the  center  of  the  town. 
They  also  receive  a  high  resolution  image  indicating  the 
recent  relocation  of  vehicle  obstacles  in  the  town  square. 
Lastly,  they  receive  another  report  that  a  ten  (10)  vehicle 
convoy  had  stopped  in  the  center  of  the  town,  fifteen  (15) 
minutes  ago. 

The  force  is  able  to  rapidly  ingest  this  information  and 
reactively  plan  and  execute.  Information  generated  by 
the  local  world  can  significantly  affect  the  global  world 
assessment  of  opposing  force  intent.  This  dismounted 
force  entering  the  town  can  provide  imagery  of  the 
building  interiors  and  report  on  their  structural  integrity 
and  fortifications.  This  force  could  be  considered  to  be  a 
probe,  determining  the  opposing  force  strength  in  an 
area.  Also,  prior  to  a  precision  strike,  organic  sensors 
such  as  an  imaging  UAV  could  be  emplaced  to  provide 
accurate  battle  damage  assessment  (BDA).  The  ground 
perspective  image  could  be  sent  to  the  firing  force  to 
help  determine  if  an  additional  strike  is  necessary. 

2.  ARCHITECTURE 

It  is  important  to  recognize  that  with  the  introduction  of  a 
local  fusion  node,  the  battlefield  fusion  process  would 
become  somewhat  symmetrical.  This  processing 
symmetry  allows  for  efficient  interaction  between  the 
local  and  global  worlds.  The  increased  local  processing 
and  subsequently  increased  available  information  also 
forces  an  increase  in  global  processing. 

Relevant  global  intelligence  information  can  be 
invaluable  to  the  dismounted  warfighter  provided  it  is 
significant  and  timely.  It  is  also  critical  to  not  burden 
these  warfighters  with  extraneous  information  or 
introduce  more  uncertainty.  Therefore,  this  information 
must  be  filtered  from  the  plethora  of  irrelevant  (to  this 
particular  warfighter)  information  generated  in  the  global 
world. 

These  filters  must  be  intelligent  [4]  enough  to  select  only 
information  that  meet  a  rigorous  acceptance  criteria 
based  on  time,  space,  mission,  certainty,  resolution, 
organization,  etc.  to  minimize  information  overload  or 
disbelief,  Infonnation  that  does  pass  these  dynamic 
intelligent  filters  must  be  transformed  to  a  representation 
that  is  readily  fused  and  consistent  with  the  available 
local  hifonnation.  Figure  2  presents  a  proposed 
architecture  for  this  global  and  local  interaction. 


Enhanced  Situational  Awareness  and  Action 


Figure  2.  Local/Global  Interaction 


Similarly  infonnation  from  the  local  world  must  be 
filtered  and  transfonned  to  be  readily  ingested  and  fused 
in  the  global  space.  This  local  infonnation  can  impact  the 
global  world  by  providing  high  resolution  ground 
perspectives,  cue  global  assets,  and  detect  local  patterns 
which  initiate  global  pattern  analysis. 


As  an  example,  consider  mortal*  attacks  on  a  fixed 
installation.  There  may  be  cell  phone  activity  just  prior  to 
firing  and  just  after  impact.  The  attacks  may  also  be 
synchronized  with  the  blue  force  patrol  activity.  This 
type  of  local  pattern  analysis  and  detection  (utilizing 
both  local  and  global  hifonnation)  would  help  military 
intelligence  discern  if  these  activities  were  repeated  in  a 
more  global  pattern. 


3.  LOCAL  FUSION 

Unlike  the  well  established  global  fusion  domain,  the 
local  fusion  concept  is  just  evolving.  Local  sensors 
included  hi  the  Waniorfs  Edge  demonstrations  included 
acoustic,  magnetic,  pomt  infrared,  visible  and  infrared 
imagers,  laser  rangefinders,  ladars,  and  seismic.  The 
global  world  can  provide  infonnation  from  a  myriad  and 
diverse  set  of  sensors  { SIGINT,  MASINT,  IMINT,  and 
HUMINT}.  The  global  world  also  produces  reports  from 
a  variety  of  analysts  and  other  sources. 

Envisioned  local  fusion  node  functionality  includes  the 
following: 

1.  Multiple  Hypothesis  Trackers 

2.  Video  Mosaicing  and  differencing 

3 .  Local/Global  Fusion  { SIGINT,  MASINT, 
IMINT,  and  HUMINT}. 

4.  Cross  Sensor  Cueing 

•  Between  Organic  Sensors 

•  Between  Local/Global  Assets 

5.  Infonnation  Kiosk/Visualization 

6.  Planning/Control  for  Organic  Sensors 

7.  Pattern  Analysis 

8.  Training 

•  After  Action  Review 

•  Simulations 
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9.  Communications  Hub 

•  Local  Node 

•  To  Global  Fusion  Node 

•  To  other  Local  Nodes 

10.  Language  Translation  Triage  (document) 

Current  processing  power  limitations  (determined  mainly 
by  electrical  power  availability)  restrict  the  use  of  a 
distributed  fusion  approach  integrated  within  the 
warfighter’s  embedded  processor  (current  PDA, 
wearable,  or  tablet  technology).  With  the  inexorable 
processing  technology  advances  such  as  mainstream 
multi-core  processors  this  restriction  should  be  rapidly 
eliminated. 

As  an  alternative  approach,  the  Future  Combat  System 
concept  envisions  the  squad/platoon  level  dismounted 
infantry  augmented  with  a  MULE  as  a  logistics  carrier  to 
help  unburden  the  infantry  from  their  Herculean  load. 
This  vehicle  should  provide  enough  electrical  power  and 
space  for  a  processing  node.  This  functionality  is 
extended  here,  by  making  the  MULE  an  information 
carrier  incorporating  an  information  fusion  station. 

The  communications  and  processing  enhancements 
embedded  in  a  local  fusion  station  also  enable  linkage  to 
other  information  nodes.  This  local  fusion  node  is 
intended  to  provide  a  broad  range  of  fusion  functionality 
as  well  as  other  useful  local  information  processing 
tasks. 


Figure  3.  Convoy  tracks  and  lines  of  bearing 

Some  examples  of  these  processes  will  be  given.  The 
multi-hypothesis  correlator/tracker  (multi-target,  multi¬ 
sensor)  fuses  the  acoustic  array  based  unattended  ground 
sensor  vectors  to  detect  and  tract  convoys  (Figure  3),  as 
well  as  to  localize  gunshot  and  other  transient  acoustic 
events  (Figure  4). 

In  both  figures  the  sensors  are  represented  by  light  blue 
dots.  The  red  icons  represent  locations  of  the  convoy. 
The  blue  lines  depict  the  lines  of  bearing  from  the 
acoustic  arrays.  The  acoustic  and  cued  imaging  sensors 


for  the  gunfire  detection  and  localization  were  mounted 
on  small  robotic  vehicles.  Both  the  lines  of  bearing  and 
either  the  visible  or  infrared  video  along  those  lines  of 
bearing  are  sent  to  the  MULE  for  fusion/localization. 

The  yellow  lines  in  Figure  3  represent  the  point  infrared 
sensors  used  to  detect  and  count  convoy  vehicles. 

Figure  4  shows  the  line  of  bearing  response  to  5 
sequential  gunshots  fired  from  the  comer  of  a  building  at 
the  McKenna  MOUT  site.  These  lines  of  bearing  are 
fused  to  provide  a  position  of  the  shooter.  The  cued 
visible  and  infrared  sensors  provide  images  from  those 
lines  to  the  warfighter. 


Figure  4.  Gunshot  lines  of  bearing 


An  example  of  information  from  global  sensors  pertinent 
to  the  local  squad  world  is  high  resolution  imagery. 
During  the  demonstration  a  high  resolution  visible  image 
chip  was  generated  and  received  from  the  global  world 
and  was  registered  and  integrated  into  the  map  display 
(ARCMAP)  of  the  dismounted  platoon  leader.  This 
image  chip  passed  the  spatial  and  temporal  filters. 
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Figure  5.  Platoon  leaders  display 
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This  Local/global  interaction  illustrated  the  need  for 
precise  registration  of  global  images  of  urban  settings  (if 
these  images  are  to  be  used  as  part  of  the  local  operating 
picture).  The  fusion  of  the  three  robotic  acoustic  arrays 
yielded  localization  accuracy  in  the  order  of  one  or  two 
meters.  The  distance  between  buildings  at  the 
demonstration  site  was  3-4  meters.  Therefore  local 
situational  awareness  requires  a  registration  in  the  order 
of  one  meter.  Figure  5  is  an  image  of  the  platoon 
leader’s  display  which  has  an  image  chip  from  a  global 
sensor  registered  with  ARCMAP. 

This  tablet  display  includes  the  location  of  the  squad 
members  and  their  robotic  assets.  The  green  lines  are  a 
use  of  “digital  ink”.  The  warfighters  can  graphically 
communicate  and  collaborate  by  drawing  symbology  and 
intensions  (i.e.,  they  can  place  standard  military 
symbology  on  the  display  and  draw  freehand,  to 
emphasize  areas  of  interest,  movement,  or  required 
action).  This  warfighter  to  warfighter  graphical 
collaboration  tool  shows  significant  promise  even  in  the 
rapid  urban  tempo. 

4.  LOCAL  PROCESSING  EXTENSIONS 

The  local  fusion  node  has  demonstrated  tremendous 
potential  to  enhance  the  situational  awareness  of  the 
dismounted  infantry  and  military  intelligence.  Its 
potential  can  be  further  increased  by  the  recognition  that 
extensions  to  its  core  processes  could  also  provide 
several  other  valuable  functions  to  the  dismounted 
infantry.  Embedded  training,  pattern  recognition,  and 
asset  control  and  management  functions  can  be 
implemented  by  extensions  to  the  local  fusion  design. 

By  archiving  the  blue  and  red  force  data  and  providing 
sophisticated  playback  options  and  analysis  functions  a 
fusion  node  could  demonstrate  the  following  embedded 
training  functionality: 

•  Platoon  After  Action  Reviews 

•  Battalion  Review  of  TTP’s 

•  Dismounted  Infantry  and  Military 
Intelligence  Cross  Training 

•  Mission  Rehearsals 

•  Military  School  Training  Updates 


Extensions  to  Local  Fusion 


Figure  6.  Local  Fusion  Extensions 

Similarly,  local  datamining  and  pattern  recognition 
functionality  (listed  below)  can  be  implemented.  This 
functionality  is  particularly  useful  in  fixed  site 
protection.  Enhanced  visualization  [5]  is  key  to  the 
warfighters  ability  to  rapidly  recognize  and  assimilate 
any  blue  and  red  patterns.  Figure  6  illustrates  the 
functions  that  can  be  enhanced  by  being  closely  coupled 
to  the  local  fusion  process. 

1 .  Playback  with  map  context  (window,  forward 
and  reverse) 

2.  Temporal  and  spatial  correlation  with  other 
events 

•  COMINT 

•  HUMINT 

3.  Enemy  activity  (disruptions  in  power, 
communications,  internet) 

4.  Background  activity  (prayer,  business,  school, 
etc.) 

5.  Blue  Force  activity  (checkpoints,  patrols, 
convoys,  etc.) 

6.  Correlation  with  terrain  features  (lines  of 
communications,  masked  areas,  etc) 

7.  Histogram  tune  of  event,  distance  to  event,  tune 
between  events 

A  multi-resolution  approach  to  visualization  [4,5]  as  well 
as  analysis  is  required.  A  global  infrastructure  with  the 
ability  to  visualize  the  battlefield  environment  (terrain, 
weather,  entities,  features,  communications,  control 
measures,  etc.)  at  various  resolutions  is  needed  in  both 
the  local  and  global  worlds.  This  enables  the 
commander  to  have  a  custom  global  view  of  the 
battlefield  as  well  as  a  high-resolution  local  view  to 
support  critical  decisions.  This  same  infrastructure 
supports  high  fidelity  local  views  for  the  dismounted 
commander  as  well  as  the  ability  to  jump  to  other  local 
views  to  support  training  or  preparation  for  deployment. 
This  scalability  provides  a  single  visualization  approach 
suitable  for  both  global  and  local  applications.  A  2D/3D 
approach  is  necessary  since  soldiers  are  very  familial* 
with  two-dimensional  maps  and  can  maintain  their  global 
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Although  this  effort  is  focused  on  the  dismounted 
infantry,  all  these  concepts  apply  to  the  mounted  domain 
as  well.  This  fusion  station  took  several  forms 
(depending  on  the  platform)  in  the  demonstrations  to 
represent  several  classes  of  users.  A  robotic  M-Gator 
represented  a  MULE  type  system.  A  HMMWV  slide  in 
fusion  station  was  utilized  for  mounted  and  dismounted 
maneuver  interaction.  Work  is  continuing  to  shrink  the 
power  and  size  of  this  fusion  station. 
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•  A  coherent  integration  of  the  organic  sensor 
information  at  the  local  level. 

•  The  ingestion  and  fusion  of  global  information 
that  is  significant  to  the  squad. 

•  An  effective  real  time  visualization  of  the 
situation. 

•  The  generation  of  ground  perspective 
information  for  the  global  world. 

The  platoon  leader  and  his  RTO  were  able  to  quickly 
react  to  this  fused  sensor  information  and  modify  the 
action  of  the  squad.  As  soon  as  mortars,  gunfire,  and 
convoys  were  detected  and  localized,  the  platoon  leader 
was  able  to  reactive ly  plan  and  transmit  this  new  plan 
graphically  to  his  squad. 

Situational  awareness  at  both  the  squad  and  global  levels 
can  be  significantly  improved  with  an  effective 
interaction  between  the  local  and  global  worlds. 
Although  not  demonstrated  in  this  year’s  exercise,  the 
ability  to  perform  an  after  action  review  based  on  the 
fusion  station’s  archived  information  would  significantly 
improve  the  warfighter’  ability  to  utilize  both  local  and 
global  information.  This  would  be  a  step  toward 
providing  a  training  environment  that  links  Intelligence 
and  Dismounted  Maneuver. 


situation  awareness.  However  the  2D  representation  is 
not  as  effective  for  high  resolution,  complex  terrain.  3D 
representation  is  excellent  for  high  resolution  complex 
terrain,  but  it  is  veiy  easy  to  lose  a  global  perspective  in 
all  the  detail  presented.  Presenting  and  coupling  both 
views  simultaneously  eliminates  many  of  the  problems 
inherent  in  a  single  view  approach.  Figure  7  illustrates  a 
coupled  2D/3D  visualization  approach. 


Figure  7.  2D/3D  Linked  Visualization 

5.  CONCLUSION 

The  Warrior’s  Edge  program  successfully  demonstrated 
the  functionality  of  a  dismounted  squad  local  fusion 
station.  This  local  fusion  station  provided  four  critical 
functions: 
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